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PREFACE 


The  work  covered  by  this  Research  Note  was  conducted  by  the  Computer 
Sciences  Laboratory  (CSL) , U.S.  Army  Engineer  Topographic  Laboratories 
(ETL) , Fort  Belvoir,  Virginia.  It  is  part  of  an  effort  being  carried  out 
in  CSL  on  digital  image  analysis  under  Project  No.  1T855T20010.  Studies 
were  conducted  by  Michael  A.  Crombie  with  computer  programming  assistance 
by  Thomas  A.  Hay  and  Charles  A.  Haase.  Samuel  Barr  performed  the 
mensuration. 
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STEREO  ANALYSIS  OF  A SPECIFIC  DIGITAL  MODEL  SAMPLED  FROM  AERIAL  IMAGERY 


i 

| 


A complete  analysis  of  digital  matching  accuracies  for  a variety  of 
images  and  taking  geometries  is  beyond  this  study.  The  need  for  such  a 
study  is  reviewed  in  the  Discussion  section  of  this  report. 

However,  several  secondary  objectives  were  achieved  during  this  study. 
A controlled  digital  stereo  pair  with  a dense  network  of  corresponding 
points  is  needed  at  ETL  for  several  in-house  studies.  The  results  of  this 
study  have  provided  a stereo  scene  on  disc  along  with  a large  number  (over 
150,000)  of  matched  points  at  approximately  (6-meter)  intervals.  These 
data  will  be  used  in  the  interactive  digital  processing  system  at  ETL  to 
continue  a semi-automatic  pass  point  mensuration  study,  to  perform  an 
image  warp  analysis,  and  to  investigate  Fourier  matching.  The  set  of 
data  can  also  be  used  in  a contour  analysis  or  possibly  in  a controlled 
feature  extraction  study. 

GEOMETRIC  DESCRIPTION.  The  purpose  of  this  section  is  to  describe 
the  digital  images  and  to  determine  corresponding  points.  The  various 
topics  in  the  matching  process  are  enlarged  upon  in  the  next  two  sections. 

Scanned  Data.  The  stereo  scene  is  a subset  of  the  stereo  model 
formed  by  two  near  vertical  images  taken  on  1 October  1966,  near 
Guadalupe,  Arizona.  The  exterior  orientation  data  is  given  in  a previous 
ETL  Research  Note.^  The  upper  left  corner  of  the  pair  was  marked  in 
stereo  prior  to  scanning  on  the  PDS  1050A  Automatic  Microdensitometer 
System.  Eight  fiducial  marks  and  their  neighboring  reseaus  along  with 
five  reseaus  over  the  approximately  25  -cm  square  subset  were  measured 

^Michael  A.  Crombie,  Philip  G.  Lem,  and  Thomas  A.  Hay,  Single  Photo 
Analysis  of  Sampled  Aerial  Imagery,  U.S.  Army  Engineer  Topographic 
Laboratories,  Fort  Belvoir,  VA. , ETL-RN-74-10 , August  1974,  AD  A012  176. 


INTRODUCTION.  The  purpose  of  this  report  is  to  describe  a stereo 
matching  analysis  made  at  the  Computer  Sciences  Laboratory  (CSL) , U.S. 

Army  Engineer  Topographic  Laboratories  (ETL).  The  primary  objective  was 
f to  measure  elevation  accuracies  of  heights  determined  by  digitally 

matching  a stereo  pair  of  photographs.  This  goal  could  not  be  met 
completely  in  the  current  effort  owing  to  the  lack  of  Imagery  with  known 
and  photo  identifiable  ground  control.  The  conventional  method  of 
computing  a manual  estimate  of  height  online  was  not  applicable  in  this 
experiment,  nor  could  a set  of  corresponding  points  be  determined  during 
the  image  scanning  because  the  PDS  I050A  microdensitometer  is  a mono 
comparator-scanner.  The  only  other  method  would  have  been  to  mark  a large 
f number  of  corresponding  points  in  stereo  prior  to  scanning.  However,  this 

procedure  would  have  removed  imagery  from  both  images  and  hampered  the 
matching  process.  For  these  reasons,  the  match  results  are  evaluated  in 
a relative  sense.  This  is  done  by  considering  Y-parallax  errors,  by 
viewing  the  shape  and  numerical  values  of  the  correlation,  and  by  review- 
ing the  average  signal  power  and  number  of  successful  matches. 
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before  and  after  the  scanning  process.  These  data  were  used  to  calculate 
a transformation  from  pixel  space  to  fiducial  space  and  its  inverse. 

The  images  were  placed  in  the  microdensitometer  comparator  so  that 
the  base  line  was  parallel  to  one  of  the  comparator  axes.  The  pixel 
spacing  was  24  ym  (Micrometer)  and  the  line  spacing  was  24  ym.  The  pixel 
diameter  was  34.5  ym;  2048  lines  and  2048  pixels  per  line  were  measured 
on  both  images.  The  microdensitometer  output  10-bit  density  data  on  tape, 
which  was  eventually  packed  ind  stored  on  disc  in  the  image  processing 
system  at  ETL. 


Pixel  Space  and  Image  Space.  Pixel  coordinates  range  from  1 to 
2048  for  both  coordinates  (I,J)  and  for  each  image.  The  point  at  (1,1)  on 
the  left  image  corresponds  to  the  point  at  (1,1)  on  the  right  image 
because  that  point  was  marked  in  stereo.  Note  that  neither  the  point  at 
(1,1)  nor  its  immediate  neighbors  are  valid  points  because  of  the  stereo 
marking  operation.  If  (I,J)  are  the  pixel  coordinates  of  any  point  P, 
then  the  comparator  coordinates  are 

Xpc  = Xo  - (J-l)  * 24 

Ypc  = Yo  + (1-1)  * 24 

The  values  (Xo,  Yo)  are  the  comparator  coordinates  of  the  marked  point. 
Note  that  the  comparator  frame  is  lefthanded  and  that  the  Y-axis  is 
toward  the  flight  direction.  The  fiducial  and  reseau  measurements  were 
used  to  develop  two  transformations  of  the  following  form: 

Xpf  ■ r Xpc  + s Ypc  + t 

Ypf  = u Xpc  + v Ypc  + w 

This  transformation  is  an  intermediate  one  that  removes  film  distortion 
but  does  not  compensate  for  radial  lens  distortion  or  air  refraction. 
Radial  lens  distortion  was  removed  by  table  look  up  and  air  refraction 
distortion  was  removed  by  using  the  simple  flat  earth  model  described 
by  Schut.  Routines  were  written  for  the  CDC  6*00  to  go  back  and  forth 
between  pixel  space  and  image  space. 

Scene  Geometry.  The  coordinate  frames  defired  above  are 
described  in  figure  1. 
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G.  H.  Schut,  "Photograrametric  Refraction"  Photogrammetric  Engineering, 
Vol.  XXXV,  January  1969. 


The  symbol  y pertains  to  the  epipolar  line  through  the  general  point  P. 

If  £5,  exaggerated  in  the  figure,  is  the  epipolar  angle  of  y with  respect 
to  the  Xp  - axis,  then  1I/2  - (£+&)  is  the  angle  y makes  with  the  Yc  and  I 
axes.  The  angle  £ was  estimated  from  the  transformation  coefficients  to 
be  £ 'v  Tan  “1  (-u/r).  If  the  letters  L and  R pertain  to  the  left  and 
right  images,  then  CL  "V  269.3°.  It  can  be  shown  that 

0.42°  £ £L  <_  0.51° 

-0.04°  1 BL  £ 0.04° 

for  the  image  subset;  then  the  epipolar  lines  yL  and  yR  through  corres- 
ponding points  PL  and  PR  are  nearly  parallel  to  the  Yc  and  I axes.  This 
was  to  be  expected,  since  the  photography  is  near  vertical  and  since  the 
Yc  (or  I)  axis  was  constrained  to  be  parallel  to  the  base  line. 

The  two  images  were  reproduced  on  the  DICOMED  at  approximately  1.7x 
enlargement.  The  grid  on  the  left  (independent)  image  is  defined  by  the 
21  horizontal  lines  IL  and  the  21  vertical  lines  JL. 

IL  = 13  + (N-l)  * 100 

JL  = 13  + (N-l)  * 100 
N = 1,  21 

The  grid  is  used  to  define  20  files  in  the  J-direction  and  20  blocks  in 
the  I-direction.  The  file  and  block  notation  was  used  as  a convenient 
method  for  describing  areas  of  interest  and  for  organizing  the  data 
reduction.  Note  that  the  scan  lines  (parallel  to  the  J-axis)  were  moved 
incrementally  in  the  positive  I-direction;  whereas,  the  DICOMED  printed 
the  images  in  the  opposite  direction. 


Matching  Procedures.  Regular  spaced  points  were  defined  on  the 
left  (independent)  image  (figure  2),  and  tne  match  process  involved 
predicting  where  the  corresponding  points  were  on  the  right  (dependent) 
image  (figure  3)  and  then  refining  the  estimates  by  using  correlation 
procedures.  The  match  process  operated  along  files  processing  one  block 
at  a time.  A modified  version  of  the  infiltration  method ^ was  used  to 
review  each  match,  and  the  correlation  function  was  developed  along  the 
epipolar  line  through  the  match  point  estimate.  All  matching  took  place 
in  pixel  space.  Several  aspects  of  the  process  are  discussed  below  and 
in  the  next  two  sections. 


Prediction  and  Shaping.  Suppose  the  independent  pixel 
coordinates  on  the  left  image  are  designated  (X,Y)  and  the  dependent 


Paul  Rosenberg,  Kent  E.  Erickson,  and  Gerhard  C.  Rowe,  Digital  Mapping 
System:  Mathematical  Processing,  U.S.  Army  Engineer  Topographic  Labora- 
tories, Fort  Belvoir , VA. , ETL-CR-74-6,  May  1974,  AD  782  230. 
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Figure  3.  Right  Image 


coordinates  on  the  right  image  are  designated  (u,v).  The  (X,Y)  values 
are  integer  values  and  were  defined  for  the  I block  and  J file  as  follows: 


Xj  = Xj  + (j-1)  * 5 
Yi  = Yj  + (i-1)  * 5 
i and  j range  from  1 to  20. 

Xj  = 16  + (J-1)  * 100 

Yj  = 16  + (1-1)  * 100 

J = 1,  20  files 
I = 1,  20  blocks 

Thus,  400  points  are  defined  in  each  of  the  400  blocks. 

It  is  assumed  that  one-to-one  transformations  u=u  (X,Y)  and  v=v 
(X,Y)  exist  and  that  the  Taylor  expansions,  up  to  but  not  including  second 
order  terms,  are  close  approximations  to  the  transformations  over  small 
regions.  Suppose  that  (Xp,  Yp)  and  (up,  vp)  are  corresponding  points, 
then 

up'  up  + du  AX  + 6_u  AY 
6X  6Y 

vp'  "v  vp  + dv  AX  + dv  AY 
6X  6Y 

where 

Xp'  = Xp  + AX 
Yp'  = Yp  + AY 

The  up'  and  vp'  equations  are  the  estimators.  The  independent  increments 
AX  and  AY  are  defined  in  a regular  fashion,  and  the  partial  derivatives 
are  estimated  from  neighboring  matches. 

A point  is  redefined  as  an  array  of  pixels  centered  over  the  point 
for  the  correlation  calculations.  The  left  array  is  shaped,  since  a 
number  of  arrays  must  be  defined  on  the  right  image  in  order  to  develop 
the  correlation  function.  The  spacing  in  the  left  array  is 
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where  in  this  instance  Au  = Av  = 1,  i.e.  uniformly  spaced  pixels  on  the 
right  image  are  used.  It  is  still  necessary  to  resample  for  gray  shades 
on  the  right  image,  but  the  number  of  computer  multiplies  is  diminished. 

Modified  Infiltration.  The  infiltration  method  is  a relatively 
complex  process  that  follows  the  path  of  easiest  correlation.  That  is, 
the  process  results  dictate  the  direction  of  the  process,  and  the  system 
evaluates  each  match  to  determine  whether  it  is  acceptable  or  not.  Thus, 
regions  of  low  signal  power  are  avoided  until  surrounding  regions  of 
better  imagery  are  matched.  The  more  difficult  regions  are  then  pro- 
cessed utilizing  all  of  the  acquired  parallax  information  or  else  are 
avoided  completely.  In  this  manner,  the  system  matches  all  of  the  model 
that  can  be  matched,  and  in  the  process  delineates  those  regions  that 
cannot  be  matched,  e.g.  ponds,  clouded  areas,  occluded  areas.  The 
infiltration  method  was  demonstrated  for  small  scenes  in  a previous  ETL 
report.^  A need  exists  for  a special  computing  capability  and  for  large 
amounts  of  readily  accessible  computer  storage  before  the  process  can  be 
used  over  an  entire  model  or  even  over  10  percent  of  one  as  in  this 
exercise.  For  this  reason,  the  following  procedure  was  tested. 

The  independent  points  for  the  blocks  are  shown  in  figure  4.  The 
last  two  rows  of  matched  results  from  the  previous  block  and  the  last  two 
columns  of  matched  results  of  the  same  block  from  the  previous  file  were 
used  to  begin  processing  the  current  block.  These  data  along  with 
matched  results  of  this  block  were  used  to  compute  the  partial  derivative 
estimates  used  for  predicting  and  shaping.  The  matching  process  (explain- 
ed below)  produced  a numerical  evaluation  of  each  match.  At  the  comple- 
tion of  the  block  run,  each  of  the  400  matches  was  compared  to  a prede- 
termined criterion.  Those  points  that  failed  were  rematched,  this  time 
using  a larger  defining  array  of  pixels  and  a more  refined  partial 
derivative  estimate. 

Evaluation.  Since  the  defining  epipolar  plane  through  the  base 
line  and  the  independent  point  (X,Y)  is  known,  the  epipolar  line  on  the 
dependent  image  can  be  easily  computed.  A correlation  function  through 
the  dependent  coordinate  estimate  (up',  vp')  and  parallel  to  the  epipolar 
line  is  generated.  A second  correlation  function  through  the  estimate 
and  perpendicular  to  the  epipolar  line  is  also  computed.  Note  that  the 
slope  of  the  epipolar  line  changes  very  slowly  over  the  model  subset. 
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Paul  Rosenberg,  Kent  E.  Erickson,  and  Gerhard  C.  Rowe,  Digital  Mapping 
System:  Mathematical  Processing,  U.S.  Army  Engineer  Topographic  Labora- 

tories, Fort  Belvoir,  VA. , ETL-CR-74-6,  May  1974,  AD  782  230. 


The  shift  along  the  epipolar  line  corresponding  to  the  peak  correlation 
is  used  to  refine  the  estimate.  The  shift  perpendicular  to  the  epipolar 
line  is  also  estimated,  and  a fraction  of  this  shift  is  applied  to  the 
estimate  as  a means  for  controlling  Y-parallax  error  build-up.  Y- 
parallax  is  a photogrammetric  term  for  the  component  of  missmatch  normal 
to  the  epipolar  line.  The  term  is  used  exactly  in  that  context  in  this 
report  even  though  the  pixel  Y-direction  is  nearly  parallel  to  the 
epipolar  direction.  Thus,  Y-parallax  errors  in  this  exercise  are  actually 
in  the  X-direction. 

A quadratic  function  was  fit  through  the  largest  of  the  discrete 
correlation  values  and  its  adjacent  neighbors.  The  shape  of  the 
correlation  curve  was  used  to  evaluate  the  match.  A narrow  correlation 
function  is  regarded  as  indicative  of  a better  match  than  a flatter  one. 
For  this  reason,  the  product  of  the  second  derivatives  of  the  two 
functions  was  used  to  evaluate  the  match.  Large  positive  values  of  the 
product  indicate  successful  matches. 

The  numerical  correlation  procedure  is  shown  in  figure  5. 


Correlation 
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In  this  example,  the  encircled  discrete  correlation  values  would  be  used 
to  estimate  the  function,  the  shift,  and  finally  the  shape  of  the  corre- 
lation function.  The  individual  correlation  value  used  in  this  exercise 
was  the  linear  correlation  coefficients  of  statistics. 


Rir  - 1 (1  ~ 1?  (r  - r)  1 

U U - I)2  1 (r  - O2]  2 

The  summation  is  taken  over  the  defining  array  where  1 and  r pertain  to 
the  gray  shades  of  the  left  and  right  images,  respectively. 

Geometric  Control.  Several  techniques  were  used  to  maintain 
geometric  control  over  the  process.  One  technique  was  described  above 
in  the  section  on  the  Modified  Infiltration  method.  Continuity  was 
maintained  over  blocks  and  files  by  using  boundary  data  of  previously 
matched  blocks  to  help  compute  the  partial  derivatives  needed  for  depen- 
dent coordinate  estimating  and  shaping.  Modifications  to  the  general 
technique  were  needed  for  File  #1  and  for  all  20  of  the  first  blocks. 

In  order  to  maintain  continuity  within  a block,  a five-point  cubic 
smoothing  function  was  applied  to  each  of  the  20  lines  after  they  were 
matched.  The  last  two  points  of  the  adjacent  block  were  included  in  the 
smoothing.  At  the  completion  of  the  block  run,  25  points  were  inter- 
sected in  object  space  to  determine  if  a significant  amount  of  Y-parallax 
had  crept  into  the  computations.  If  the  Y-parallax  estimate  exceeded  a 
a test  criterion  (5pm  in  this  exercise),  then  each  of  the  400  points  were 
corrected.  This  was  done  by  allowing  all  of  the  intersection  error  to  be 
taken  up  by  the  dependent  coordinates  in  image  space.  The  reason  for  this 
is  that  the  independent  coordinates  are  correct  by  definition.  The 
derived  transformation  from  image  space  to  pixel  space  was  applied  to  the 
image  space  corrections  to  derive  the  appropriate  dependent  pixel  space 
corrections. 

NUMERICAL  EXPERIMENT.  The  purpose  of  this  section  is  to  describe 
some  of  the  preliminary  tests  and  operations  that  were  performed  on  the 
digital  image  pair  and  also  to  describe  the  results  of  the  final  computer 
run.  Matching  parameters  determined  in  the  preliminary  tests  were  used  to 
specify  the  matching  procedure  of  the  final  computer  run. 

Preliminary  Tests.  Prior  to  matching  a large  number  of 
corresponding  points  over  the  digital  model,  two  series  of  tests  were 
performed.  The  first  series  were  analytical  tests  that  utilized  the 
geometry  of  the  imagery  to  obtain  bounds  on  the  shape  parameters  and  on 
the  expected  parallax  changes.  The  second  series  were  numerical  tests 
designed  to  evaluate  the  analytical  results  and  to  compute  practical 
matching  parameters. 

Analy t ical . Scale  change  determination  for  image  segments  along 
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epipolar  Lines  for  changing  slopes  is  critical  to  correct  image  shaping 
and  to  estimate  precise  match  points.  The  estimate  of  the  match  points 
must  be  is  accurate  as  possible  so  as  to  reduce  the  dimension  of  the 
correlation  function  in  the  refinement  process.  One  purpose  of  this 
analysis  was  to  compute  reasonable  upper  and  lower  bounds  of  the  stretch 
parameter  so  that  the  sample  estimate  could  be  constrained.  It  was 
expected  that  the  matched  data  would  be  subject  to  noise,  which  is  magni- 
fied in  the  process  of  computing  sample  derivatives.  The  equation  used 
to  compute  the  results  below  is  contained  in  appendix  A. 

The  scale  change  parameter  is  the  ratio  of  corresponding  image  seg- 
ments for  a particular  ground  segment.  Since  the  shape  parameter  decreases 
as  the  location  parameters  w and  £ (see  appendix  A)  increase,  table  1 
pertains  to  £ = w = 0.  The  table  pertains  to  slopes  at  the  nadir  of  the 
first  image  and  to  a base-height  ratio  of  B/H  = 0.6. 

a S 


0 

1.000 

5 

1.055 

10 

1.118 

15 

1.192 

20 

1.279 

25 

1.389 

30 

1.530 

35 

1.725 

40 

2.014 

45 

2.500 

Table  1.  Maximum  Scale  Change  Parameters 


The  angle  a is  expressed  in  degrees  and  is  the  slope  of  the  terrain  in 
object  space.  Note  that  the  results  pertain  to  a Segment  tilted  toward 
image  2.  If  the  slope  is  in  the  opposite  direction,  then  S is  the  recip- 
rocal of  the  values  given  above.  If  the  slope  is  less  than  25°  in 
at^bolute  value,  then  0.7  <_  S <_  1.4.  In  the  section  on  Scene  Geometry 
above,  the  angle  between  the  epipolar  line  and  the  Y-axis  was  less  than 
/>ne  degree.  For  this  reason,  = Vy  was  constrained  to  lie  between 


6Y 


1.4  and  0.7.  In  order  to  account  for 
differences  between  the  ideal  and  the 
were  also  applied: 

0.995  < — = U 

- 6X  X 

6U  = U 

-0.005  < <*Y 


nonlinearity  in  the  scanner  and 
actual,  the  following  constraints 


< 1.005 

< 0.005 
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£V 

-0.005  < fix  = Vv  < 0.005 

Control  was  maintained  on  any  one  of  the  four  shape  parameters  (say 
W)  in  two  ways: 

1.  Compute  running  average 

2.  Enforce  constraints 

The  running  average  is  simply 

WI+1  “ 6 * W i + (1-6)  * WQ 
where 

Wl+i  • updated  value 

Wj  : previous  value 

: sample  value 

6 : running  average  weight;  o < B < 1 

This  calculation  only  applies  if  the  sample  value  was  computed  in  the 
imnediate  neighborhood  of  the  previous  value.  The  updated  value  is 
constrained  to  lie  within  the  boundaries  given  above. 

For  a variety  of  reasons,  such  as  inadequate  shaping,  poor  match 
point  estimation,  lack  of  image  detail,  occluded  scenes,  it  is  very 
possible  that  the  process  could  lock  onto  the  wrong  point  especially 
when  the  matching  arrays  are  small.  Thus,  reasonable  bounds  must  be 
placed  on  the  parallax  corrections  that  are  directly  related  to  changes 
in  terrain  slope.  Table  2 presents  expected  pixel  shifts  as  a function 
of  slope  c.  ange  Aa  and  pixel  spacing  M.  The  entries  pertain  only  to  the 
digital  image  model  analyzed  in  this  exercise. 

It  was  assumed  in  the  derivation  of  table  2 that  the  terrain  had 
been  flat  when  an  abrupt  change  of  Aa  occurred.  The  entries  in  table  3 
are  multiplicative  scale  factors  that  can  be  used  to  increase  the 
expected  parallax  shifts  of  table  2 whenever  the  original  slope  was  a. 

It  is  evident  that  any  pixel  shift  larger  than  one  spot  should  be  suspect. 

Numerical . The  initial  computation  was  a fishing  expedition  to 
determine  a starting  point  for  subsequent  computations.  The  pixel 
located  at  JX  = IY  * 16  on  the  left  image  was  chosen  arbitrarily,  and  a 
coarse  matching  procedure  was  initiated  at  and  around  UJ  = VI  = 16.0  on 
the  right  image.  A two-dimensional  correlation  function  was  generated 
by  allowing  VI  to  vary  in  increments  of  one  pixel  from  VI  = 11.0  and 
continuing  until  VI  = 40.0.  Since  the  I-axis  is  nearly  parallel  to  the 
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Table  2.  Parallax  Shift  Due  to  Slope  Changes. 


epipolar  direction,  UJ  was  allowed  to  vary  in  increments  of  one  pixel 
from  UJ  = 14.0  to  UJ  = 1S.C.  There  was  no  shaping,  the  linear  corre- 
lation coefficient  was  used,  and  the  array  size  was  21  x 21.  The  peak 
correlation  was  determined  by  inspection  from  the  150  correlation  values 
to  be  located  at  UJ  - 16.0  and  VI  = 32.0.  Thus,  the  following 
coordinate  pair  in  pixel  space  were  regarded  as  good  starting  values  in 
subsequent  computer  runs: 


Left  Image 


Right  Image 


JX 

IY 


16 

16 


UJ 

VI 


16.0 

32.0 


A variety  of  computer  runs  were  performed  over  Block  1 (Bl)  of  File 
1 (FI)  as  a means  to  get  reasonable  parameters  for  the  subsequent  pro- 
cessing of  the  digital  model.  The  process  described  in  figure  4 could 
not  be  used,  since  the  required  starting  data  did  not  exist.  The  first 
step  was  to  match  the  four  points  in  figure  6. 
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MN 

11 

10° 

11° 

O 

o| 

CM 

25°  1 

1 

.01 

.03 

.04 

.05 

.07 

2 

.03 

.05 

.08 

.11 

.14 

3 

.04 

.08 

.12 

.16 

.21 

4 

.05 

.11 

.16 

.22 

.28 

5 

.07 

.13 

.20 

.27 

.35 

6 

.08 

.16 

.24 

.33 

.42 

7 

.09 

.19 

.28 

.38 

.49 

8 

.10 

.21 

.32 

.44 

.56 

9 

.12 

.24 

.36 

.49 

.63 

10 

.13 

.26 

.40 

.54 

.70 

Table  3. 

Parallax 

Shift  Scale 

Factors . 

*£3 

11 

IS 

o 

11° 

20° 

25° 

5° 

1.02 

1.03 

1.10 

1.17 

1.27 

10° 

1.02 

1.06 

1.12 

1.21 

1.33 

15° 

1.03 

1.08 

1.15 

1.25 

1.39 

20° 

1.04 

1.10 

1.19 

1.31 

1.47 

25° 

1.05 

1.12 

1.22 

1.36 

1.56 

r 


Left  Image 


Right  Image 


Figure  6.  Initial  Pattern. 

The  output  pixel  spacing  M on  the  left  image  was  one  of  the  parameters 
that  was  varied.  The  corresponding  points  on  the  right  image  were 
estimated  by  assuming  zero  parallax.  For  example,  after  point  A was 
refined,  the  coordinates  of  point  B were  estimated  to  be 

UJB  = UJA 

Vlg  = VIA  + M 

The  refinement  process  consisted  of  developing  LC  correlation  values  along 
the  epipolar  line  through  the  estimate  and  also  LC  correlation  values 
normal  to  the  epipolar  line  and  of  then  correcting  the  estimate  by  the 
shift  as  described  in  figure  5.  The  number  of  correlation  values  LC,  the 
form  of  the  correlation  measure,  the  array  size,  and  the  resampling 
procedure  along  with  the  output  spacing  M are  some  of  the  parameters  that 
were  analyzed  in  the  preliminary  tests. 


Twenty  rows  and  twenty  columns  of  matched  points  were  processed  in 
the  several  preliminary  tests.  The  remaining  396  points  were  matched 
over  the  pattern  in  figure  7. 


Path  1 Path  2 Path  18 

AC*  * • OOO  • 


• B D • 


.E  F 


Figure  7.  Preliminary  Test  Point  Path. 

Local  parallax  information  was  used  for  both  predicting  and  array  shaping. 
For  example,  the  partial  derivatives  for  points  E and  F were  computed 
from  the  following  equations: 

6U  = L.  (UC  " UA  + ud  - V 

6X  2M 

fill  = i_  (vc  - vA  + vD  - V ) 

6Y  2M 

§1  = L_  % ' UA  + UD  " UC> 

6X  2M 

— = — <VB  - VA  + VD  - VC) 

6Y  2M 


The  shape  parameter  estimates  were  more  complicated  as  the  point  in 
question  moved  away  from  the  boundary.  Also,  the  running  average  notion 
and  the  constraints  described  above  were  utilized. 


The  objective  of  the  preliminary  numerical  tests  was  to  determine 
a sensible  set  of  match  parameters  for  the  digital  imagery  used  in  this 
experiment.  It  appears  that  such  a study  is  required  whenever  new 
imagery  is  tested.  The  following  parameters  were  evaluated  for  the 
initial  tests  on  B1  of  FI: 


IC 

LC 

M 

RES 

CM 

YP 


Array  Size 

Length  of  Correlation  Function 
Output  Spacing 
Resampling  Method 
Correlation  Measure 
Y-Parallax  Correction 


The  specific  goals  were  to  minimize  IC  and  LC,  to  maximize  M,  and  to 
simplify  RES,  CM,  and  YP.  One  restraint  on  the  matching  procedure  was 
that  the  digital  data  was  st.-^ed  on  disc,  had  to  be  extracted  from 
disc  memory,  and  put  into  core  memory  before  processing.  The  parameter 
selection  was  tempered  by  the  limits  on  core  memory,  and  the  time 
required  to  transfer  data  between  disc  memory  and  core. 


The  array  size  IC  was  varied  from  7 to  19  pixels,  and  the  length  of 
the  correlation  function  LC  was  varied  from  5 to  9 pixels.  The  output 
spacing  M was  varied  from  1 to  5 pixels.  The  two  correlation  measures 
RXY  and  SXY  were  compared,  and  the  bilinear  interpolation  and  nearest 
neighbor  resampling  procedures  were  compared.  The  RXY  and  SXY  measures 
are  defined  as 


RXY  = oxy/axay 


SXY  = a 


xy 


The  symbols  o. 


and  ox,  and  a are  the  covariance  of  X and  Y,  the 


Xy ’ “““  “x’  “y 

standard  deviation  of  X,  and  the  standard  deviation  of  Y,  respectively. 

The  two  arrays  of  pixels  are  symbolized  as  X and  Y.  Finally,  the 
Y-parallax  correction  was  evaluated  at  e=0.01,  0.05  and  0.1  times  the 
computed  shift  in  the  direction  normal  to  the  epipolar  direction. 


There  were  400  matches  for  each  of  the  approximately  20  computer 
runs.  An  extensive  correlation  history  was  printed  out  for  each  match. 
The  output  included  the  correlation  function  values,  the  estimated 
X-parallax  shifts,  the  shape  parameters,  signal  power,  confidence 
measure,  and  finally  the  Y-parallax  error  for  each  of  the  matching 
paths  described  in  figure  7.  The  two  basic  measures  of  effectiveness 
were  the  Y-parallax  errors  and  the  computed  X-parallax  shifts.  The 
X-parallax  shifts  were  compared  to  table  2.  X-parallax  is  a photo- 
grammetric  term  for  the  component  of  missmatch  along  the  epipolar 


20 


direction.  Thus,  X-parallax  errors  in  this  exercise  are  actually  in  the 
pixel  Y-direction. 


The  tightest  control  on  Y-parallax  (e=0.01)  produced  the  best  results. 
The  primary  reason  for  this  is  that  the  epipolar  directions  were  known 
accurately.  However,  Y-parallax  must  still  be  controlled  for  two  reasons. 
First,  the  inaccuracies  in  the  prediction  parameters  will  cause  errors  in 
Y-parallax.  Second,  nonlinearities  in  the  scanner  will  also  cause 
Y-parallax  errors. 

Several  general  observations  were  made  as  the  output  spacing  M and 
the  array  size  IC  were  varied. 

1.  The  match  results,  except  for  M = 1,  generally  improved 
as  IC  increased. 

2.  The  match  results  definitely  worsened  for  M = 1 as  IC 
increased . 

3.  Except  for  IC  = 7 , results  generally  improved  as  M increased. 

4.  The  match  results  definitely  worsened  for  IC  = 7 as  M 
increased . 

5.  Although  there  was  an  improvement  in  output  quality  as  IC 
increased  from  11  to  19  for  M > 1,  the  results  were  not 
nearly  as  significant  as  the  change  from  IC  = 7 to  IC  ■ 11. 

6.  The  match  results,  except  for  M = 1,  definitely  worsened 
for  IC  = 7 as  the  length  of  the  matching  path  increased. 

The  first  two  observations  indicate  that  when  the  spacing  M is 
small  with  respect  to  the  array  size  IC,  then  first  order  shaping  can 
easily  produce  incorrect  results.  This  is  true  because  the  computed 
shape  parameters  pertain  to  a small  percentage  of  the  array.  As  M 
increases  with  respect  to  IC,  the  estimated  shape  parameter  is  more 
nearly  the  average  of  the  array. 

The  third  and  fourth  observations  indicate  that  when  the  spacing  is 
large  with  respect  to  the  array  size,  then  again  first  order  shaping 
can  easily  produce  incorrect  results.  Again  this  is  true  because 
the  estimated  shape  parameter  pertains  to  or.ly  a small  percentage  of 
the  array.  For  example  if  M = 5 and  IC  = 7,  then  no  portion  of  the 
previously  estimated  match  points  used  to  update  the  shape  parameters 
are  i uded  in  the  central  array.  When  M = 5 and  IC  = 11,  the  central 
just  barely  overlaps  the  previously  matched  data. 

The  four  observations  taken  together  are  not  contradictory.  They 
indicate,  for  the  parameter  range  considered  in  the  tests,  that  there 
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is  an  optimal  set  somewhere  between  the  extremes.  The  output  spacing 
M = 5 was  chosen  to  reduce  the  cjmputation  time  and  to  reduce  the 
amount  of  storage.  The  array  size  was  set  to  IC  = 11  because  of  the 
fifth  observation.  In  order  to  take  advantage  of  the  improvement  in 
output  as  IC  is  increased,  IC  was  set  to  19  for  the  rematch  procedure 
described  in  the  section  on  Modified  Infiltration. 

The  sixth  observation  indicates  that  if  an  inappropriate  set  of 
matching  parameters  are  chosen,  then  the  errors  are  likely  to  propagate 
unfavorably.  It  would  be  very  uncertain  to  rely  on  correlation  methods 
to  recover  lost  correlation,  unless  a strategy  is  developed  to  do  so. 

From  tables  2 and  3 and  for  M = 5,  the  largest  computed  shift  along 
the  epipolar  line  owing  to  a change  in  ground  slope  should  be  less  than 
0.6  pixel.  Since  the  shape  and  prediction  parameters  are  estimated  and 
therefore  contain  errors,  the  computed  pixel  shift  can  be  as  large  as 
one  pixel  spacing  ar  even  a bit  larger  when  traversing  steep  areas  of 
low  contrast.  For  these  reasons,  the  number  of  correlation  values  LC 
should  not  exceed  5,  i • e.  two  values  on  either  side  of  the  estimate. 

In  fact,  the  match  results  definitely  got  worse  as  LC  increased  from 
5 to  9. 

It  was  shown  in  an  ETL  Research  Report^  that  the  linear  correlation 
coefficient  RXY  is  superior  to  the  covariance  SXY  in  stereo  image 
matching.  It  was  shown  again  in  this  preliminary  numerical  analysis 
that  RXY  is  indeed  superior  to  SXY,  especially  in  areas  of  low  contrast. 
Finally,  there  was  a significant  improvement  in  match  quality  when  the 
pixels  of  the  shaped  arrays  were  determined  by  interpolation  rather 
than  by  nearest  neighbor.  Although  RXY  generally  takes  twice  the 
compute  time  of  SXY  and  interpolation  takes  longer  to  compute  than 
nearest  neighbor,  it  was  felt  that  the  increase  in  compute  time  was  well 
worth  the  effort  since  fewer  strategies  would  be  needed  in  steep  areas 
and  or  in  areas  of  low  contrast. 


Stereo  Processing  of  the  Stereo  Image  Pair.  The  digital  imagery 
was  processed  one  file  at  a time.  The  first  block  (Bl)  of  each  file 
was  processed  first  by  tying  it  to  Bl  of  the  previous  file.  The  tie  was 
implemented  by  using  the  last  two  columns  of  results  of  the  prior  block 
for  developing  of  shape  parameters,  for  predicting,  and  for  smoothing. 

The  remaining  blocks  in  the  file  were  processed  sequentially;  wherein, 
the  last  two  columns  of  the  corresponding  block  in  the  previous  file 
and  the  last  two  rows  of  the  prior  block  in  the  current  file  were  used 
to  maintain  continuity.  The  first  block  of  the  first  file  (FI)  was 
processed  independently  of  the  neighboring  blocks.  The  remaining  blocks 

^Michael  A.  Crombie,  Semi-Automatic  Pass  Point  Determination  Using 
Digital  Techniques,  U.S.  Army  Engineer  Topographic  Laboratories, 

Fort  Belvoir , VA. , ETL-0051,  December  1975,  AD  A026  082. 
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of  FI  were  processed  sequentially;  wherein,  the  last  two  rows  of  results 
from  the  previous  block  were  used  to  maintain  continuity. 


An  attempt  to  match  the  400  output  points  of  each  block  was  made  in 
Mode  1 of  the  operation.  Those  points  not  meeting  the  success  criterion 
were  rematched  in  Mode  2.  If,  during  Mode  1,  the  confidence  measure 
Cd>N  _>  0.01,  then  the  match  was  judged  to  be  a success;  C<)>N  is  the  pro- 
duct of  the  two  second  derivatives  of  the  two  correlation  functions 
associated  with  each  match.  The  array  size  in  Mode  1 was  IC  = 11  and 
in  Mode  2 it  was  IC  * 19.  The  following  set  of  parameters  was  used  in  the 
match  process: 


Length  of  Correlation  Function 
Output  Spacing 
Correlation  Measure 
Y-Parallax  Correction 
Running  Average  Weight 
Y-Parallax  Criterion 


LC  = 5 pixels 
M = 5 pixels 
RXY 

e = 0.01 

e = 0.50 

6 = 5 pm 


If  the  sample  estimate  of  the  Y-parallax  error  exceeded  6,  then  all  400 
points  were  corrected  for  Y-parallax.  The  bilinear  interpolation  for 
gray  shades  was  used  in  the  resampling  operation,  and  the  shape  para- 
meters were  constrained  by  the  following  bounds: 

0.995  <UX  < 1.005 

-0.005  < Vx  < 0.005 

-0.005  £ Uy  <_  0.005 

0.700  < Vy  < 1.400 

Numerical  Results.  The  computer  program  was  organized  so  that 
only  enough  pixel  data  was  extracted  from  disc  to  process  one  block  at 
a time.  The  pixel  data  dimension  of  the  independent  image  subset  was 
constant  for  each  block.  The  pixel  data  from  the  dependent  image  varied 
over  blocks  owing  to  the  undulating  terrain.  The  initial  line  of  pixel 
data  on  disc  for  the  dependent  image  was  estimated  accurately  by  re- 
viewing the  dependent  coordinates  of  the  last  line  of  output  data  from 
the  previous  block. 

Not  all  of  the  first  blocks  could  be  matched  in  their  entirety, 
since  the  dependent  pixel  data  did  not  extend  for  enough  in  the  minus 
Y-direction  beginning  approximately  at  F3.  Line  10  of  B1  for  all  20 
files  was  the  first  completely  matched  line.  Thus,  there  are  400  * 

391  = 156,400  matched  points  over  the  digital  model. 

The  basic  outputs  from  the  process  are  the  dependent  pixel  coordi- 
nates on  the  second  image  associated  with  the  regularly  spaced  inde- 
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pendent  coordinates  on  the  first  image.  These  results  were  stored  on 
disc  into  two  (400  x 400)  regions.  If  the  independent  pixel  coordinates 
on  the  first  image  are  defined  to  be 

X = 16  + (J-l)  * 5 

Y = 16  + (1-1)  * 5 

I = 10,400  and  J = 1,400 

then  the  dependent  pixel  coordinates  on  the  second  image  are  stored  on 
disc  in 


U = DEPX  (I,J) 
V = DEPY  (I,J) 


where 


DEPX  and  DEPY  are  the  areas  on  disc 
reserved  for  the  dependent  data. 

Thus,  the  156,400  corresponding  pairs  of  points  are  defined  by  (I,J)  in 
the  first  case  and  located  on  disc  by  (I,J)  in  the  second  case. 

The  origin  of  a local  rectangular  coordinate  frame  was  located  at 
pixel  coordinates  X^  = Y = 1011  of  the  independent  image.  The 
corresponding  pixel  coordinates  on  the  dependent  image  are  located  at 
U<j>  = DEPX  (200,000)  and  V ^ = DEPY  (200,000j.  These  data  were  trans- 
ferred to  image  space,  corrected  for  distortion,  and  then  intersected 
in  the  earth  fixed  geocentric  reference  frame.  The  geocentric  coordi- 
nates of  the  origin  were  used  to  establish  a local  coordinate  frame; 
wherein,  the  Xl  YL-plane  is  tangent  to  the  Clarke  1866  spheroid  at  the 
nadir  of  the  intersected  point  and  where  the  XL-axis  is  toward  the 
north.  Finally,  the  remaining  156,399  points  were  intersected  in  the 
local  frame,  and  the  resulting  rectangular  coordinates  were  stored  in 
three  areas  on  the  same  disc  as  (U,V).  The  local  coordinates  were 
stored  in 


XL  = XX  (I,J) 

XL  = YY  (I,J) 

ZL  = ZZ  (I,J) 

The  results  of  the  match  exercise  are  characterized  numerically  by 
the  four  tables  given  below.  The  results  are  enlarged  upon  in  the 
Discussion  section.  The  computer  program  was  designed  to  produce  the 
average  correlation,  the  average  signal  power,  and  the  average  confi- 
dence measure  for  each  block.  The  average  correlation  for  a block  is 
defined  to  be: 
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Table  4.  Average  Correlation  Values 


f 


FILE  NUMBER 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

77? 

.743 

.614 

.567 

.573 

.457 

.440 

.503 

.562 

.504 

7P  3 

.757 

.726 

.704 

.671 

.621 

.636 

.597 

.690 

.676 

78? 

.7,5 

• 68  4 

.690 

. 745 

.732 

.647 

.637 

.685 

.746 

8 31 

.fl”  7 

. 734 

.750 

.823 

.832 

.752 

.741 

.798 

.746 

774 

. 7 7 u 

.802 

.725 

. 740 

.757 

. 859 

.888 

.869 

.855 

740 

.779 

.791 

.828 

.775 

.792 

.793 

.828 

.743 

.733 

675 

.697 

. 767 

.810 

.779 

.801 

. 736 

.758 

.754 

.811 

6 P 9 

.610 

.775 

.307 

.772 

.776 

. 754 

.788 

.749 

.725 

731 

.547 

.62  6 

.6  39 

.803 

.757 

.690 

.624 

.621 

.687 

616 

.622 

.60  8 

.660 

.593 

.783 

.682 

.710 

. 714 

.742 

735 

.589 

.584 

.558 

.631 

.757 

.732 

.725 

.734 

.78  6 

7 G 2 

.618 

.482 

.663 

.536 

.520 

.657 

.762 

.721 

.722 

730 

.653 

.496 

.6  26 

.479 

.565 

.485 

.487 

.766 

.737 

769 

.632 

.496 

.719 

.474 

.594 

.584 

.565 

.667 

.740 

b&9 

• 60  4 

.543 

.623 

.510 

.692 

.565 

.5  96 

.642 

.677 

588 

.553 

.517 

.636 

.56  7 

.630 

.688 

.502 

.653 

.666 

555 

.657 

.55  0 

.5  96 

.678 

.592 

.607 

.713 

.687 

.824 

615 

.672 

.566 

.567 

. 741 

.754 

. 685 

.722 

.797 

.738 

65? 

.628 

.476 

.493 

.767 

.781 

.784 

.482 

.533 

.720 

785 

.734 

,bS7 

. 77? 

. 830 

.657 

. 736 

.619 

.659 

.713 

L 
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Table  6.  Average  Confidence 


FILE  NUM3E9 


1 

•> 

3 

4 

5 

6 

7 

9 

9 

10 

06 '« 

.0*0 

.016 

.024 

.039 

.029 

.032 

.0  34 

.054 

.029 

07? 

.063 

.051 

• P 29 

.035 

.033 

.049 

.039 

.050 

.051 

Qfcf 

.053 

. 044 

.057 

.073 

.051 

. 041 

.031 

.045 

.048 

037 

.044 

. 040 

.0  33 

.041 

.049 

. 05  2 

.049 

.044 

.049 

04? 

.046 

.053 

.037 

.033 

.031 

.043 

. 036 

.034 

.057 

* 

0 3*» 

.042 

.034 

.034 

.043 

.039 

. 039 

.034 

.035 

.046 

u ?7 

.029 

. 034 

.033 

.041 

.037 

.034 

.033 

.033 

1 

.032 

03' 

.021 

.033 

.0  25 

.034 

.039 

. 031 

.0  30 

.029 

.023 

0 3? 

.014 

.024 

.023 

.042 

.041 

. 030 

.031 

.026 

.030 

04K 

.944 

.032 

.020 

.030 

.035 

. 035 

.040 

.040 

.036 

039 

.0  39 

.034 

.039 

.032 

.0  34 

. 033 

.044 

.052 

.039 

0*43 

.033 

.031 

. G 36 

.031 

.022 

. 035 

.039 

.041 

.049 

04* 

.036 

. 03  0 

.031 

.026 

.029 

.031 

.031 

.039 

• 045 

047 

.035 

. 04  2 

.041 

.039 

.057 

.056 

.044 

.036 

.036 

047 

.036 

. 044 

.053 

.037 

.060 

. 06  3 

.091 

.069 

.038 

05P 

.037 

.042 

. 063 

. 057 

.046 

. 066 

.041 

.061 

.057 

U 34 

• G26 

.026 

.059 

.069 

.051 

.041 

.051 

.050 

.045 

Q 23 

.029 

.030 

.0  35 

.039 

.040  , 

. 043 

.037 

.040 

.055 

02« 

.026 

.022 

.023 

.031 

.045 

. 046 

.0  26 

.024 

.041 

024 

.025 

.022 

.023 

.026 

.025 

.031 

.023 

.017 

.025 

29 

— — — — _ _ 

L. 
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Table  7 


Total  Successful  Matches 


fill  number 


1 

2 

3 

4 

5 

6 

7 

3 

9 

10 

350. 

297. 

179. 

276. 

277. 

268. 

259. 

2*8. 

278. 

274 

4 00. 

389  . 

564  . 

283. 

284. 

327. 

375. 

386. 

392. 

389 

388. 

39*.. 

364  . 

391. 

400. 

372. 

346. 

339. 

393. 

38Q 

38(3. 

357  . 

390. 

551. 

319. 

385. 

3 99. 

385. 

353. 

382 

379. 

395  . 

3 85  . 

545. 

361. 

3*3. 

380. 

322. 

323. 

369 

3b9. 

382. 

382. 

374. 

390. 

358. 

351. 

368. 

364. 

38? 

3.31. 

519. 

352. 

370. 

392. 

372. 

350  . 

325. 

307  , 

336 

322. 

258. 

560. 

330  . 

348. 

36  0. 

3 59. 

363. 

333. 

304 

333. 

240. 

321. 

286. 

380. 

383. 

339. 

36  2. 

327. 

319 

365. 

32  4. 

304. 

242. 

326. 

37  3. 

367. 

356. 

370. 

379 

185. 

366. 

343. 

364. 

343. 

355. 

372. 

38  0. 

396. 

373 

37*. 

345  . 

312  . 

379. 

332. 

269. 

369. 

374. 

385. 

397 

394. 

330  . 

508. 

355. 

2 84. 

289. 

302. 

329. 

363. 

390 

393. 

306. 

303. 

384. 

315. 

392. 

392. 

362. 

386. 

373 

389. 

302  . 

298. 

387. 

315. 

38  3. 

3 96. 

399. 

382. 

370 

376  • 

317. 

290. 

394. 

358. 

386. 

3 92. 

324. 

397. 

375 

3 27. 

*87. 

?7  7. 

390. 

391. 

385. 

371. 

386. 

385. 

387 

251. 

279. 

260. 

351. 

371. 

36  8. 

362. 

37  5 

373. 

385 

234. 

242. 

22  7. 

258. 

333. 

365. 

356. 

31  5. 

290. 

373 

285. 

265. 

249. 

293. 

299. 

30  0. 

323. 

286. 

226. 

265 

31 


Table  7.  (Cont'd) 


FILE  NUM8FR 


It 

12 

lb 

14 

15 

16 

17 

18 

19 

20 

283. 

294  . 

262. 

2 A3  • 

291. 

300. 

319. 

297. 

298. 

292 

390. 

3 96  . 

393. 

38  A , 

390. 

398. 

400. 

400. 

400. 

394 

3 93. 

335  . 

361. 

30  3. 

393.  • 

395. 

397. 

40  0 . 

398. 

400 

390. 

387  . 

3 A U • 

374  . 

389. 

369. 

400  . 

40  0. 

400. 

377 

374. 

375. 

377. 

369. 

3A9. 

394. 

400. 

40U. 

394. 

391 

*71. 

22c . 

257. 

312. 

360  . 

393. 

385. 

396. 

351. 

317 

3 u 9. 

293  . 

3 34. 

335. 

363  . 

357. 

378. 

368. 

391. 

394 

25°. 

212  . 

280. 

345. 

243. 

310. 

2 98. 

319. 

348. 

356 

291. 

353  . 

2 56 . 

326. 

293. 

341. 

356. 

287. 

396. 

400 

2j*. 

374  . 

365. 

37  7 . 

370  . 

349. 

353. 

334. 

374. 

400 

2 3*;. 

359. 

372. 

394. 

363. 

349. 

371. 

376. 

386. 

388 

3i5. 

36o  . 

569 . 

365. 

317. 

347. 

359. 

32  2. 

250. 

356 

3 39. 

j3». 

395. 

369. 

391. 

344. 

387. 

344. 

291. 

385 

3?r. 

362. 

3 97. 

3A4. 

318  . 

352. 

314. 

343. 

344. 

318 

33". 

364  • 

369. 

387. 

346. 

343. 

302. 

3 '9. 

369. 

3b0 

J3*  . 

347. 

3 AO  . 

366. 

326. 

3b6 . 

162. 

254. 

321. 

397 

3L  1 , 

321. 

22°. 

219. 

320. 

292. 

387. 

366. 

365 

3 3f  . 

375. 

271  . 

263. 

214. 

246. 

348  . 

36  0. 

364. 

358 

34». 

352. 

c aa. 

27?. 

190, 

26  2. 

287. 

275. 

339. 

330 

224. 

243  . 

109. 

22  * , 

166. 

216. 

104. 

170. 

179. 

240 

i 
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r = 400  1 - ^rEp  + rNP ) 
p=l  2 

The  symbols  r^p  and  r^p  are  the  peak  correlations  in  the  epipolar  direction 
and  in  the  direction  normal  to  the  epipolar  direction.  The  average  signal 
power  for  a block  is  defined  to  be 


SP  = 1 
400 


The  symbol  a*  is  the  variance  of  the  pixel  data  from  the  left  image. 

Note  that  ox2  is  the  generally  accepted  definition  for  signal  power.  The 
average  confidence  measure  for  a block  is  defined  to  be 


where  C$N  pertains  to  the  Mode  1 match  if  the  match  was  adjusted  success- 
ful in  Mode  1 or  else  C$Np  pertains  to  Mode  2.  The  N2  values  pertain  to 
the  total  number  of  successful  matches  where  the  success  criterion  is 
C$Np  _>  0.01. 

DISCUSSION.  The  six  topics  referred  previously  are  enlarged  upon  in 
this  section,  the  topics  are: 

1.  Match  Success  Parameters 

2.  Shaping  Experiment 

3.  Array  Algebra  Experiment 

4.  General  Need  for  Tuning  Test 

5.  Utilization  of  Digital  Stereo  Model 

6.  Digital  Mapping  Advantages 

The  first  three  topics  pertain  to  the  general  digital  matching  pro- 
cedures used  in  this  report,  especially  the  dependent  and  independent 
images.  The  matching  techniques  employed  for  this  study  are  different 
from  the  conventional  methods  used  in  such  devices  as  UNAMACE,  GESTALT, 
and  AS-ll-BX  in  that  points  to  be  matched  are  defined  on  one  image 
(independent  image)  rather  than  defined  in  a model  space.  There  were  two 
reasons  for  doing  this.  The  first  reason  is  a practical  one  in  that  only 
one  image  array  need  be  shaped;  whereas,  both  images  must  be  shaped  if 
the  point  is  defined  in  a model  space.  The  second  reason  is  that  the 
digital  image  matching  effort  at  CSL  is  directed  toward  an  interactive 
match  procedure  where  it  is  often  necessary  to  select  points  for 
matching  from  digitized  images  displayed  on  a TV  screen.  The  fourth  and 
sixth  topics  result  from  the  experience  gained  in  the  study,  and  the 
fifth  topic  pertains  to  one  of  the  objectives  of  the  study. 
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Match  Success  Parameters.  A simple  scheme  was  employed  in  this 
study  to  predict  whether  or  not  a match  was  successful.  A match  was 
considered  successful  of  C(#>N  >_  0.01  where  C$N  is  the  product  of  the  two 

second  derivatives  of  the  two  correlation  functions.  Further  studies 
should  be  made  to  determine  whether  the  average  correlation  r,  the 
signal  power  SP,  and  the  shift  in  X-parallax  PK  can  be  included  for  a 
more  precise  evaluation.  A specific  combination  of  C4>N , r,  and  PK  was 
used  in  the  K+E  study.  The  graph  shown  in  figure  8 demonstrates  the 
relationship  among  the  three  parameters  r,  SP,  and  N£  where  N£  is  the 
number  of  successful  matches.  The  graph  pertains  to  Block  1 through  20 
of  File  7.  File  7 was  chosen  because  it  was  the  subject  of  the  runoff 
test  for  shaping  versus  nonshaping  described  in  the  next  section  and  in 
Appendix  B. 

It  is  apparent  from  the  graphs  in  figure  8 that  N2,  the  number  of 
successful  matches  by  definition,  follows  a trend  over  blocks  set  out 
by  both  r and  SP.  The  signal  power,  SP,  is  measured  before  the  match 
and  can  therefore  be  used  to  predict  the  likelihood  of  a successful 
match  or  else  to  select  a match  strategy.  The  average  correlation 
value,  r,  is  measured  during  the  match  process  and  can  therefore  be 
used  to  evaluate  the  match.  For  example,  the  signal  power  and  the 
average  correlation  from  B13  are  low,  which  should  indicate  that  more 
work  will  be  required  to  meet  a specific  match  accuracy  for  points  in 
the  block  than  those  of  B5  where  the  two  measures  were  high. 

Note  that  N£,  the  number  of  successful  matches,  is  larger  for  B4 
than  B5,  yet  SP  and  r were  lower.  When  the  information  content  is  very 
high,  as  in  B5,  the  correlation  curve  tends  to  flatten  out  rather  than 
display  a sharp  peak.  This  means  that  for  large  values  of  SP  the 
matching  array  dimensions  should  be  diminished,  or  else  a more  sophis- 
ticated shape  function  should  be  employed. 

Shaping  Experiment.  Simple  first  order  shaping  as  described 
previously  in  Geometric  Description  was  used  in  the  match  process.  The 
purpose  of  the  experiment,  described  in  appendix  B and  enlarged  upon 
here,  was  to  demonstrate  that  array  shaping  is  necessary  as  higher  order 
accuracy  requirements  are  placed  on  the  output.  This  was  done  by 
comparing  a nonshaping  run  to  one  where  the  arrays  were  shaped.  One  of 
the  rationales  for  using  digital  techniques  is  that  complex  shaping  can 
be  used  if  necessary;  whereas,  complex  raster  shaping  in  analog 
methods  is  impractical.  More  refined  array  shaping  can  be  accomplished 
in  several  ways.  One  possible  method  that  offers  many  advantages  is 
discussed  in  the  next  section. 

Blocks  B2  to  B19  of  F7  were  used  in  the  test.  The  topography  is 
displayed  in  figure  2,  and  the  average  slope  and  standard  deviation  of 
the  slopes  are  given  below  in  table  8.  The  average  slope  is  the  average 
of  the  X-slopes  and  Y-slopes  where  all  possible  slopes  were  computed  for 
each  block.  The  units  in  table  8 are  degrees. 
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Table  8.  Slope  Data  for  File  7. 


Block 

Slope 

Standard 

Deviation 

Block 

Slope 

Standard 

Deviation 

2 

12.2 

7.9 

11 

10.1 

6.2 

3 

13.6 

9.5 

12 

10.7 

7.5 

4 

9.3 

4.2 

13 

13.9 

9.7 

5 

9.2 

8.0 

14 

11.8 

5.4 

6 

11.8 

7.5 

15 

7.3 

4.8 

7 

7.5 

5.8 

16 

4.7 

4.1 

8 

5.5 

4.7 

17 

6.0 

4.9 

9 

8.4 

6.2 

18 

5.2 

4.5 

10 

9.1 

5.7 

19 

4.0 

3.4 

The 

graph  shown 

in  figure  9 

demonstrates  the 

relationship 

among  the 

three  quantities  SL,  and  AN£.  The  average  slope  per  block,  SL,  is 
taken  from  table  8;  the  numerical  values  for  au  and  AN2  were  taken  from 
table  B3.  The  value  is  the  standard  error  (in  meters)  of  the  differ- 
ence between  the  shaping  and  nonshaping  run.  The  value  AN2  is  the 
increase  in  the  number  of  successful  matches  when  the  matching  arrays 
are  shaped.  The  graph  pertains  to  Block  2 through  19  of  File  7. 

It  is  apparent  from  the  graphs  in  figure  9 that  AN2  and  follow 
a trend  over  blocks  set  out  by  SL.  As  the  average  slope  increases,  the 
standard  error  in  the  elevation  computation  increases.  The  number  of 
acceptable  matches  also  increases  as  the  average  slope  increases  when 
the  matching  arrays  are  shaped.  The  results  of  the  experiment  demon- 
strate that  array  shaping,  even  the  simple  method  employed  in  this  study, 
offers  a worthwhile  improvement  in  elevation  precision  and  in  the  number 
of  acceptable  matches. 

From  the  observations  given  in  the  Preliminary  Tests  Section,  it  is 
apparent  that  the  simple  first  order  shaping  employed  in  this  study  is 
only  applicable  if  the  array  size  is  small  and  if  the  match  point  spacing 
is  small.  Generally,  if  the  match  point  spacing  is  large,  then  the 
predicted  parallax  is  less  accurate,  and  if  the  array  is  large,  the 
first  order  approximation  becomes  less  realistic.  It  is  also  apparent 
from  the  results  of  this  section  and  appendix  B that  array  shaping  under 
the  conditions  used  in  this  study  is  effective  in  computing  elevations. 

The  shaping  and  predicting  process  is  fundamentally  an  exercise  in 
extrapolation,  which  is  usually  a hazardous  operation.  This  is  especially 
true  if  the  extrapolated  points  are  far  removed  from  the  prediction  base. 
The  purpose  of  the  correlation  process  is  to  verify  and  refine  the 
prediction.  If  the  extrapolation  is  over  long  distances,  then  the 
correlation  process  becomes  less  realistic. 
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There  are  at  least  three  ways  to  overcome  the  numerical  difficulties 
described  above.  The  first  is  to  match  many  points,  perhaps  more  than  is 
needed  for  the  task,  such  as  was  done  in  this  study.  The  second  method  is 
to  use  the  basic  matching  process  described  in  this  study  but  with  larger 
arrays  and  larger  spacings  and  also  to  use  higher  order  shaping.  The 
errors  in  extrapolation  still  will  not  go  away  because  the  parameters 
needed  for  second  or  higher  order  shaping  must  be  estimated  by  differencing 
parallax  data  even  further  removed  from  the  predicted  point  than  was  used 
in  first  order  shaping. 

The  third  method  is  to  concentrate  on  subregions,  wherein  points  are 
developed  in  an  iterative  fashion  over  the  subregion.  The  match  points 
are  used  to  develop  a warp  function,  which  in  turn  is  used  as  a predicting 
point  and  as  a shaping  tool.  The  process  is  considered  complete  for  a 
subregion  whenever  corrections  to  the  predicted  points  are  considered 
insignificant;  the  corrections  are  computed  using  correlation  methods. 

In  this  manner,  higher  order  shaping  and  predicting  can  be  used  if  a 
practical  means  for  developing  a warp  function  can  be  demonstrated.  A 
method  for  doing  this  is  discussed  in  the  next  section. 

Array  Algebra  Experiment.  Array  algebra^  as  discussed  here  and 
in  appendix  C pertains  to  the  efficient  adjustment  of  one  member  of  the 
family  of  multilinear  forms,  namely  the  bilinear  form.  The  bilinear  form 
is  defined  in  appendix  C as  a multiplicative  model  representation  of  P 
X-f unctions  and  Q Y-function  where  the  dependent  variable  is  say  Z.  The 
model  is  linear  in  the  P*Q  undetermined  coefficients  and  it  can  be  shown 
that  the  array  algebra  solution  is  identical  to  the  least  squares  solution. 

The  number  of  calculations  and  required  computer  storage  are  far 
less  than  for  the  conventional  least  squares  solution.  For  example,  the 
solution  time  for  the  256  parameter  model  discussed  in  appendix  C took 
about  6 seconds  on  the  CDC-6400  at  CSL.  Sophisticated  coding  and  better 
data  management  could  reduce  the  computer  time  considerably.  The  same 
problem  solved  in  the  conventional  manner  would  take  over  3 hours. 

Such  a huge  savings  in  compute  power  does  not  occur  without  con- 
straints on  the  mensuration  exercise.  The  only  constraint,  other  than 
linear  independence  among  the  unknown  coefficients,  is  that  the  data 
must  be  measured  in  the  XY-frame  such  that  the  N X-values  and  the  M 
Y-values  form  a regular  grid,  where  N>P  and  M>Q. 

^V.A.  Rauhala,  "New  Solutions  for  Fundamental  Colloction  and  Triangulation 
Problems,"  Svensk  Lantmateritidskrif t 4,  Presented  in  ISP  Congress,  Comm 
III,  Ottawa. 
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Five  scenes  from  the  digital  image  shown  in  figure  2 were  selected  to 
aid  in  the  evaluation  of  the  array  algebra  methods.  The  five  scenes  are 
enlarged  and  shown  again  in  figure  Cl.  Five  bilinear  models  were  defined 
and  fitted  to  256  of  the  2116  elevation  points  from  each  scene.  Each 
scene  is  about  275  by  275  meters  on  the  ground.  One  72-parameter  model 
was  defined  and  used  to  warp  the  five  dependent  image  scenes  to  their 
corresponding  independent  scenes.  A statistical  summary  of  the  terrain 
roughness  for  the  five  scenes  is  given  in  table  Cl.  A statistical 
summary  of  the  image  quality  and  match  quality  as  described  in  the  study 
is  given  below.  The  entries  in  table  9 were  taken  from  tables  4,  5,  6 
and  7 . 


Table  9.  Image  and  Match  Statistics  For  the  Five  Scenes 


Scene 

SP 

r 

cj>N 

n2 

1 

301 

.727 

.045 

374 

2 

169 

.530 

.041 

346 

3 

256 

.715 

.039 

356 

4 

272 

.734 

.061 

387 

5 

180 

.565 

.025 

276 

The  first  series  of  tests  were  performed  to  demonstrate  how  array 
algebra  methods  could  be  used  in  terrain  modelling.  The  25  variances 
associated  with  the  five  models  for  the  five  scenes  are  given  in  table 
C2.  The  model  standard  error  is  about  1 meter  or  less  for  the  first  four 
scenes  for  all  five  models.  The  large  variances  for  Scene  5 are  attri- 
buted to  the  uncertain  matching  process  caused  by  the  low  signal  power 
and  high  relief.  Note  that  since  Model  5 is  composed  of  256  parameters, 
the  exact  number  of  measurements  used  in  the  fit,  then  the  variance 
estimates  are  necessarily  zero.  Residuals  were  computed  for  the  1860 
points  not  used  in  the  fit  for  Model  5.  A statistical  summary  of  the  fit 
is  given  in  tables  C3  and  C4. 

The  statistical  summaries  show  that  the  bilinear  form  can  be  used 
successfully  in  a variety  of  terrain  model  exercises.  For  example,  if 
match  points  are  defined  on  a grid  in  model  space,  then  the  bilinear 
model  can  be  used  in  compilation.  If  the  elevation  data  is  represented 
by  a bilinear  form  (not  necessarily  the  examples  of  this  study),  then  its 
coefficients  can  be  developed  in  an  iterative  process  where  the  function 
itself  is  used  to  predict  match  points  and  to  perform  complex  image 
shaping.  The  function  in  this  case  is  a terrain  model,  and  contour 
segments  can  be  developed  along  with  the  compilation  process  and  error 
detection. 

The  second  series  of  tests  were  performed  to  demonstrate  how  array 
algebra  methods  could  be  used  in  image  warping.  A statistical  summary 
of  the  warp  model  results  is  given  in  table  C5.  The  model  standard 
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error  is  about  0.7  pixel  or  less  for  the  first  four  scenes,  and  as  in  the 
terrain  model  tests,  the  standard  error  for  Scene  5 is  large  for  the  same 
reasons  as  before.  A 99  percent  confidence  interval  was  constructed,  as 
before,  by  computing  all  2116  residuals. 

It  is  apparent  from  the  statistical  summaries  that  the  bilinear  form 
can  be  used  successfully  in  a variety  of  image  warp  operations.  For 
example,  if  match  points  are  defined  on  a grid  on  one  image  (as  was  done 
in  this  study),  then  the  bilinear  form  can  be  used  in  compilation.  If  the 
dependent  pixel  data  is  represented  by  a bilinear  form,  then  its  coeffi- 
cients can  be  developed  in  an  interactive  manner  where  the  function  is 
used  for  predicting,  error  detecting,  and  shaping. 

General  Need  For  Tuning  Test.  A large  portion  of  the  effort  of 
this  study  was  directed  toward  the  determination  of  a practical  set  of 
match  parameters  for  the  final  computer  run.  The  derived  set  of  parameters 
pertains  only  to  the  specific  digitized  model.  If  photographic  records 
were  collected  with  a different  camera  under  different  taking  conditions, 
then  the  set  of  parameters  used  in  this  study  would  not  pertain.  In  fact, 
if  the  imagery  used  for  this  study  were  scanned  differently  (different 
pixel  size  and  spacing),  then  again  the  set  of  derived  parameters  would 
not  necessarily  pertain. 

What  is  needed  is  a study  to  provide  operational  guidelines  for  the 
proper  selection  of  mensuration  and  matching  parameters  for  successful 
digital  photogrammetric  operations  on  photographic  imagery.  Photo- 
grammetric  operations  include  pass  point  selection,  target  determination, 
line-of-sight,  and  other  mensuration  exercises  as  well  as  compilation. 

The  basic  problem  in  computer  costs  and  output  accuracies  associated  with 
digital  photogrammetry  is  determining  practical  and  accurate  means  for 
matching  corresponding  points  on  two  or  more  stereo  images.  This  problem 
pertains  whether  the  operation  is  an  interactive  one,  as  in  target 
determination  or  map  revision,  or  whether  an  exotic  sensor  record  is  to  be 
compiled  on  a conventional  compilation  device.  Such  a study  will  provide 
operational  guidelines  that  will  eliminate  the  usual  trial  and  error 
methods  employed  when  new  imagery  is  introduced. 

There  are  several  obvious  sensor  record  parameters  that  will  tend  to 
characterize  the  imagery.  In  photographic  imagery,  base-height  (B/H), 
scale  (H/f),  and  final  image  resolution  (lp/mm)  come  immediately  to  mind. 

It  can  be  stated  that  correlation  is  easiest  when  B/H  = 0 and  that  it 
becomes  more  difficult  as  B/H  increases.  On  the  other  hand,  elevation 
accuracy  increases  as  B/H  increases;  elevations  cannot  be  computed  when 
B/H  = 0.  It  would  appear  that  lp/mm  must  be  considered,  especially  in 
relation  to  B/H.  For  example,  a taking  system  that  produces  large  values 
of  lp/mm  will  also  produce  compilation  problems  when  B/H  is  large.  The 
type  of  sensor  record,  i.e.  panoramic,  frame,  or  infrared,  will  have  a 
great  bearing  on  operating  characteristics  and  achievable  accuracies. 
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Match  performance  will  also  be  a function  of  terrain  characteristics. 

Some  of  the  obvious  parameters  are  expected  slopes,  forest  areas,  urban 
areas,  featureless  areas,  occluded  areas.  Much  of  this  can  be  monitored 
somewhat  during  compilation.  For  example,  signal  power  is  a good  indica- 
tion of  image  detail.  Very  low  values  indicate  lack  of  detail,  and  large 
values  indicate  a busy  scene.  Image  matching  tends  to  become  difficult 
at  either  extreme. 

An  analysis  of  imagery  and  of  the  taking  parameters,  when  compared 
to  the  output  requirements,  will  provide  numerical  constraints  on  the 
scanner  operation  and  on  the  match  procedure.  Scanner  characteristics 
include  spot  size  and  shape,  spot  spacing,  signal-to-noise  ratio,  density 
accuracy,  quantization  level,  comparator  precision.  The  parameters  of 
match  procedure  include  patch  size,  match  spacing,  correlation  measure, 
and  a variety  of  strategies  and  auxiliary  parameters.  These  include 
epipolar  geometry,  scale  differences,  resolution  differences,  and  image 
shaping. 

Utilization  of  Digital  Stereo  Model.  One  of  the  objectives  of 
this  study  was  to  develop  on  disc  memory  at  ETL  a digitized  stereo  model 
complete  with  a large  number  of  match  points,  a corresponding  set  of 
rectangular  local  coordinates,  and  a well-defined  relation  between  the 
pixel  coordinates  and  the  corresponding  undistorted  photographic 
coordinates.  These  results  will  be  used  in  the  following  exercises  in 
digital  processing: 

1.  Semi-Automatic  Pass  Point  Mensuration 

2.  Match  Refinement  Tests 

3.  Evaluation  of  Fourier  Matching 

4.  Image  Warp  Tests 

5.  Target  Transfer  Tests 

The  pass  point  work  is  a continuation  of  an  ETL  study  on  the  inter- 
active determination  of  corresponding  points  on  overlapping  digital 

Th*.  pilnt*!  determined  in  tl.ls  e tudy  will  be  used  as  control  for 
the  pass  point  study,  and  array  algebra  methods  will  be  used  in  areas  of 
high  relief.  In  addition,  the  derived  image  control  and  array  algebra 
methods  will  be  used  to  perform  image  warp  tests,  target  transfer  tests, 
and  the  match  refinement  tests.  The  image  warp  testr  will  be  similar  to 
those  performed  in  this  study,  and  the  match  refinement  tests  will  be 
performed  over  troublesome  regions  such  as  Scene  5 described  in  appendix 
C.  The  evaluation  of  Fourier  matching  involves  converting  phase  shifts 
of  low  frequency  harmonics  between  corresponding  digital  signals  into  a 
linear  shift  in  pixel  space. 

Digital  Mapping  Advantages.  It  became  more  apparent  during  the 
course  of  the  study  that  digital  techniques  offer  at  least  three  advantages 
o\er  analog  methods,  they  are: 
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1.  Accuracy 

2.  Flexibility 

3.  Growth  Potential 


By  utilizing  software,  output  accuracy  can  range  from  gross  accuracies  to 
the  ultimate  accuracy  that  can  be  extracted  from  the  imagery.  The 
flexibility  of  the  digital  computer  was  emphasized  during  the  preliminary 
tests  where  a variety  of  ideas  were  examined.  Concepts  are  easy  aid 
relatively  inexpensive  to  test  in  digital  computers.  It  is  much  easier 
to  discard  or  modify  a poor  matching  concept  when  the  concept  is  couched 
in  software  than  when  it  is  imbedded  in  hardware.  The  freedom  provided 
to  the  analyst  by  the  digital  computer  in  the  conceptual  design  will  also 
provide  for  increased  accuracy.  For  example,  array  algebra  techniques 
provide  the  digital  analyst  a tool  that  is  denied  the  analog  analyst. 

Not  only  is  the  capability  of  growth  potential  enhanced  by  the 
flexibility  of  software  but  new  computer  concepts  such  as  special  purpose 
digital  processors  and  parallel  processing  can  be  integrated  into  existing 
computer  centers  more  effectively  than  can  they  be  integrated  with  analog 
devices.  Studies  in  digital  image  processing  are  being  conducted  by  ETL 
in  these  areas.  It  was  shown  in  an  in-house  effort  that  the  Goodyear 
Aerospace  Corporation  STARAN  associative  array  processor  can  produce  the 
correlation  values  needed  for  matching  in  times  comparable  to  hardwired 
devices. ^ A digital  cartographic  study  is  being  performed  at  Control 
Data  Corporation  to  develop  a hardware  benchmark  constructed  of  special 
purpose  digital  processors  to  determine  if  the  new  technology  is  practical 
in  terms  of  speed,  accuracy,  and  cost. 

Digital  image  processing  will  become  even  more  advantageous  as  mass 
memories  become  a reality  and  as  image  data  is  collected  digitally.  The 
new  computer  technologies  coupled  with  mass  memories,  high  speed  data 
rates,  solid  state  array  scanners,  and  new  processing  techniques  such  as 
array  algebra  will  provide  image  processing  capabilities  that  cannot  be 
achieved  by  analog  methods. 

CONCLUSIONS 

1.  A digital  model  of  a stereo  scene  of  approximately  2,450  by  2,450 
square  meters  was  developed  and  stored  on  disc  at  CSL  for  further  studies 
in  digital  mapping.  The  model  includes  the  digitized  image  pair,  159,600 
match  points,  and  their  local  rectangular  coordinates. 

2.  Digital  matching  techniques  offer  accuracy,  flexibility,  and  growth 
potential  capabilities  not  found  in  conventional  analog  methodologies. 

7 

David  L.  Ackerman,  Michael  A.  Crombie  and  Mary  L.  Powers  Image 
Correlation  on  A Parallel  Processor.  U.S.  Army  Engineer  Topographic 
Laboratories,  Fort  Belvoir,  VA. , ETL-00  April  1976. 
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3.  Planning  and  operating  guidance  in  the  proper  selection  of  mensuration 
and  matching  parameters  is  needed  for  successful  digital  photogrammetric 
operations  on  stereo  photographic  images. 

4.  Array  algebra  methods  should  be  included  in  any  rethinking  of  digital 
mapping  or  digital  processing  in  general. 

5.  Array  shaping  is  necessary  to  achieve  accurate  image  correspondence 
in  areas  of  high  relief. 

6.  Y-parallax  errors,  if  excessive,  are  easy  to  monitor  and  simple  to 
correct  using  digital  techniques. 
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APPENDIX  A.  Scale  Change  For  A Straight  Line  Segment  On  Vertical 
Photography  As  A Function  of  Slope 


Consider  the  following  diagram.  The  objective  here  is  to  compute  the 
ratio  of  the  image  segments  on  exposures  #1  and  it 2 that  correspond  to  the 
ground  point  segment  AB. 


X = H Tanw 
A 

YA  = D 


H Tanw 


zA  = 0 


AB 

Yfi  » D + 6 


zB  = h 


The  ratio  is  simply  S = where  is  the  image  length  of  the  segment 

AB  on  //I  and  d£  is  the  image  length  of  the  segment  AB  on  //2. 


Xbl>*  + (Val  " Ybl> 


d2  =*<Xa2  " Xb2)  + <Ya2  - Yb2> 

The  image  coordinates  are  derived  directly  from  the  central  projection 
equations,  where  the  two  orientation  matrices  are  the  identity  matrices 
of  order  three: 


xal 

= f 

Tanw 

Xa2 

= f 

Tanw 

yai 

= f 

D 

ya2 

= f 

(D-B) 

H 

H 

Xbl 

= fH  Tanw/ (H-h) 

II 

CM 

>e 

Ybl 

= f 

(D  + 

6) /(H-h) 

yb2  = 

Let 

6 = 

h 

cot  a 

and 

D = 

H 

Sec  w Tan  / 

9 

Then  by  direct  substitution  and  after  a bit  of  algebraic  simplification 

2 2 I 

d^  = _fh  [Tan  w + (Cot  a + Sec  w Tan  £)  ] 2 
H-h 


d„  = fh  [Tan2  w + (H  - cot  a.  - Sec  w Tan  O ] 2 


2 


M'' 


The  desired  ratio  is 


uTan^  oj  + (cot  a 
» Tan^  W + (B  - c 


+ Sec  u>  Tan  £)' 


The  value  of  S decreases  as  both 


H 

w and 


cot  a - Sec  w Tan  £) 
£ increase. 


APPENDIX  B.  Array  Shaping  Test 


The  objective  of  this  experiment  was  to  demonstrate  the  improvement 
in  the  match  process  when  simple  first  order  shaping  is  used  compared  to 
nonshaping.  Blocks  2 to  19  of  File  7 were  rerun;  ’./herein,  the  primary 
difference  between  the  two  runs  was  that  the  following  shape  parameters 
were  used  in  the  second  run: 

Ux  = VY  - 1.0 

uY  = vx  = °.° 

That  is,  the  process  was  the  same  in  every  way  except  for  shaping.  In 
fact,  shaping  parameters  were  calculated  at  each  step  so  that  the  match 
point  could  be  estimated  more  precisely. 

The  entries  in  tables  B1  and  B2  below  were  extracted  directly  from 
the  two  computer  run  outputs.  The  column  headings  and  No  pertain  to 
the  number  of  points  successfully  matched  at  the  completion  of  modes  1 
and  2 respectively;  o and  r pertain  to  the  Y-parallax  estimates  and  to 
the  average  correlation  for  the  block;  and  finally  SP  and  c pertain  to 
the  average  signal  power  and  average  confidence  value. 

There  is  little  to  choose  from  between  the  two  runs  with  respect  to 
o and  SP.  In  the  first  case  (o) , the  apparent  equivalence  of  the 
Y-parallax  errors  indicates  that  ignoring  shaping  does  not  introduce 
Y-parallax  error.  The  signal  (3P)  in  fact  was  lower  for  the  shaping  run. 
This  is  because  the  process  of  shaping  introduces  more  dependence  among 
the  gray  shades,  which  necessarily  reduces  the  variation  and  is  equiva- 
lent to  lowering  the  signal  power. 

Table  B3  lists  the  differences  between  the  total  number  of  points 
successfully  matched  (AN2  * N2  [S]  - N2  [NS]),  the  differences  betwee.i 
the  correlation  values  (Ar  = r [S]  - T [NS]),  and  the  differences 
between  the  confidence  values  (AC  = C [S]  - C [NS]).  The  computer  pro- 
gram also  produced  the  average  difference  (e)  in  X-parallax  between  the 
two  runs  for  each  block  and  the  standard  error  (aE)  for  each  block.  The 
standard  error  in  X-parallax  was  converted  into  a standard  error  (o^  in 
meters)  in  elevation  for  each  block. 


Table  B2.  F7  Results  (Shaping) 


Block 

fl 

£ 

r 

SP 

C 

2 

307 

375 

4.2 

.636 

200 

.049 

3 

310 

346 

5.3 

.647 

211 

.041 

4 

376 

399 

1.4 

.752 

308 

.052 

5 

336 

380 

4.0 

.859 

368 

.043 

6 

327 

351 

3.9 

.793 

273 

.039 

7 

312 

350 

1.3 

.736 

229 

.034 

8 

312 

359 

5.5 

.754 

253 

.031 

9 

287 

339 

5.6 

.690 

172 

.030 

10 

324 

367 

3.9 

.682 

199 

.035 

11 

314 

372 

9.8 

.732 

211 

.033 

12 

329 

369 

5.1 

.657 

225 

.035 

13 

251 

302 

3.6 

.485 

143 

.031 

14 

348 

392 

5.  ‘ 

.584 

216 

.056 

15 

367 

396 

4.4 

.565 

201 

.063 

16 

374 

392 

9.8 

.688 

249 

.066 

17 

313 

371 

4.4 

.607 

214 

.041 

18 

317 

362 

10.5 

.685 

344 

.043 

19 

334 

356 

6.0 

.784 

307 

.046 

49 


Table  B3 


Block  Differences 


Block 

an2 

Ar 

AC 

c 

qh 

2 

22 

.049 

.006 

.113 

.609 

1.25 

3 

51 

.063 

.012 

.101 

.634 

1.28 

4 

20 

.066 

.013 

-.001 

.321 

0.64 

5 

22 

.026 

.002 

.111 

.398 

0.82 

6 

0 

.043 

.004 

.062 

.367 

0.73 

7 

9 

.032 

.000 

-.030 

.301 

0.61 

8 

3 

.008 

-.001 

.038 

.206 

0.43 

9 

14 

.021 

.001 

-.038 

.284 

0.58 

10 

16 

.036 

.003 

.003 

.238 

0.49 

11 

44 

.031 

.003 

-.006 

.280 

0.58 

12 

37 

.034 

.003 

.109 

.467 

0.95 

13 

63 

.055 

.010 

.356 

1.230 

2.50 

14 

28 

.068 

.017 

-.009 

.443 

0.91 

15 

3 

.026 

.004 

-.025 

.244 

0.49 

16 

0 

.031 

.000 

-.038 

.213 

0.43 

17 

2 

.019 

-.002 

-.013 

.246 

0.49 

18 

0 

.013 

.001 

.022 

.170 

0.34 

19 

0 

.011 

-.001 

-.004 

.111 

0.21 

50 


If  the  results  are  examined  by  block  and  visually  compared  to  File 
7 shown  on  figure  2,  then  it  is  apparent  that  shaping  is  most  effective 
in  steep  areas  and  not  particularly  effective  in  the  flatter  areas.  In 
fact,  the  shape  parameters  for  perfectly  flat  areas  are  exactly  those 
given  above.  The  average  X-parallax  difference  (e)  turns  out  to  be 
0.042  pixels,  which  is  about  1 micrometer.  This  indicates  that  ignoring 
shaping  does  not  introduce  an  X-parallax  bias.  The  average  X-parallax 
standard  error  (ae)  turns  out  to  be  0.450  pixels,  which  is  about  1 meter. 
This  indicates  that  ignoring  shaping  does  introduce  a lack  of  precision 
in  the  elevation  determinations.  It  is  expected  that  the  accuracy  and 
precision  would  diminish  even  more  if  the  shaping  parameters  were  not 
used  to  estimate  the  match  point. 
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APPENDIX  C.  Array  Algebra  Application 


Introduction.  A specific  example  of  a multilinear  form  is  presented 
as  an  example  of  how  array  algebra  can  be  used  as  a tool  in  digital 
mapping.  The  specific  form  is  the  bilinear  form  defined  as 

h (X,Y)  = f (x)  A gT  (Y) 

where 

f (X)  : (1XP)  vector  of  X-functions 

g (Y)  : (1XQ)  vector  of  Y-functions 

A : (PXQ)  array  of  coefficients 


Suppose  h (X,Y)  is  measured  at  the  NXM  intersections  of  the  orthonormal 
grid  formed  by  N X-values  and  M Y-values.  The  NXM  equations  can  be 
organized  into  the  following  matrix  equation: 


H = FAGT 


where 


F: 

(NXP) 

array 

G: 

(MXQ) 

array 

A: 

(PXQ) 

array 

H: 

(NXM) 

array 

of  X-functions 
of  Y-functions 
of  coefficients 
of  measurements 


The  objective  of  the  bilinear  solution  process  is  to  determine  the 
PXQ  coefficients.  Assume  that  N >_  P and  that  the  rank  of  F is  P.  Assume 
also  that  M > Q and  the  rank  of  G is  Q.  Multiply  the  matrix  equation  on 
the  left  by  F^  and  on  the  right  by  G to  get 

FT  H G = FtF  A GtG 

t -1 

Multiply  this  equation  on  the  left  by  (FXF)  and  on  the  right  by 
(GtG)  to  get  the  array  solution  for  A. 

A = (FtF)  -1  Ft  HG  (GtG)  -1 

This  solution  is  identical  to  the  least  squares  solution.  The  array 
solution  involves  inverting  a PXP  matrix  and  a QXQ  matrix;  whereas,  the 
least  squares  solution  requires  that  PQ  x PQ  matrix  be  inverted. 


i 
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Application.  A general  bilinear  form  solution  was  coded  in  FORTRAN 
for  the  CDC-6400. 

f U)  = [P  CO.  S CO,  c CC)] 

where 

P (£) : (LXPZ)  vector  of  powers  of  £ 

S (O : (1XPS)  vector  of  sine  functions  of  £ 

C (O : (1XPC)  vector  of  cosine  function  of  £ 

The  trig  functions  are  multiples  of  the  fundamental  period,  where  the 
fundamental  period  is  defined  over  the  range  of  the  £-data. 

Data  Set.  Five  data  sets  were  extracted  from  the  set  of  local 
Z-coordinates  on  disc.  Each  data  set  represents  approximately  a 275  by 
275  square-meters  area  on  the  ground,  or  approximately  2.25  blocks  by 
2.25  blocks.  The  five  scenes  can  be  reviewed  on  figure  2 of  section  II. 
The  northwest  block  for  each  of  the  five  scenes  is  given  next,  and  5X 
enlargements  of  each  scene  is  presented  in  figure  Cl. 

There  are  a total  of  46  by  46  elevations  in  each  set  where  256  of 
the  elevations  were  used  in  the  bilinear  fits.  The  spacing  in  each  set 
is  20  feet  (6  meters);  whereas,  the  spacing  of  the  data  used  in  the 
bilinear  fit  is  60  feet  (18  meters).  A statistical  summary  of  each  data 
set  is  given  in  the  following  table.  The  statistics  were  derived  using 
all  46  by  46  = 2116  points. 

Table  Cl.  Terrain  Statistics  (in  meters) 


1 

2 

Scene 

3 

4 

_5 

h 

442.7 

418.9 

375.6 

406.3 

443.9 

°h 

12.2 

16.7 

1.9 

2.1 

17.7 

hmax 

480.9 

455.7 

386.9 

413.6 

482.3 

hmin 

421.1 

389.1 

368.8 

401.0 

394.0 

Range 

59.8 

66.6 

18.1 

12.6 

88.3 

Here, 

is  regarded 

as  a measure 

of  spread  about  the 

mean  elevation  h 

and  not  as  an  error. 
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Figure  Cl.  Test  Scenes 


Scene 

1 

B3  of  F3 

Scene 

2 

B13  of  F7 

Scene 

3 

B15  of  F14 

Scene 

4 

B3  of  F14 

Scene 

5 

B8  of  F12 

Surface  Model.  The  two  defining  functions  f and  g were  set  equal 
in  form  for  this  specific  test.  An  analysis  should  be  made  to  determine 
what  is  the  best  form  for  general  terrain  modeling.  Five  models  (m) 
were  tested  in  this  experiment;  note  that  m=f=g  here. 

Model  1 


m (z)  - 1,  z, 

Sin  Fz,  Sin  2Fz,  Sin  3Fz,  Sin  4Fz, 
Cos  Fz , Cos  2Fz,  Cos  3Fz,  Cos  4Fz 

P = Q = 10;  100  parameters 


Model  2 


2 

m (z)  = 1,  z,  z , 

Sin  Fz,  Sin  2Fz,  Sin  3Fz, 

Cos  Fz,  Cos  2Fz,  Cos  3Fz,  Cos  4Fz 

P = Q = 10;  100  parameters 


Model  3 

m (z)  = 1,  z. 

Sin  Fz,  Sin  2Fz,  Sin  3Fz,  Sin  4Fz,  Sin  5Fz, 
Cos  Fz,  Cos  2Fz,  Cos  3Fz,  Cos  4Fz,  Cos  5Fz 

P = Q = 12;  144  parameters 


Model  4 

m (z)  = 1,  z. 


Sin 

Sin 

Fz, 

6Fz 

Sin 

• 

2Fz , 

Sin 

3Fz , 

Sin 

4Fz , 

S in 

5Fz 

Cos 

Cos 

Fz, 

6Fz 

Cos 

2Fz , 

Cos 

3Fz , 

Cos 

4Fz , 

Cos 

5Fz 

P = Q = 14;  196  parameters 

Mode  1 5 

m (z)  = 1,  z. 
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Sin 

Sin 

Fz,  Sin  2Fz,  Sin 
6Fz,  Sin  7Fz, 

3Fz , 

Sin 

4Fz , 

Sin 

5Fz , 

Cos 

Fz,  Cos  2Fz,  Cos 

3Fz , 

Cos 

4Fz , 

Cos 

5Fz, 

Cos  6Fz,  Cos  7Fz 
P = Q = 16;  256  parameters 

Note  that  the  input  data  (16  by  16  * 256)  satisfied  Model  5 exactly; 
therefore,  the  residuals  for  this  model  will  be  exactly  zero.  Residuals 
(for  Model  5)  were  also  computed  for  the  (46  by  46  - 16  by  16)  = 1,860 
points  not  used  in  the  surface  fit. 

Model  Error.  The  program  produces  the  (PxQ)  parameter  estimates, 
the  (NxM)  residuals,  and  the  model  variance  estimate.  The  variance 
estimate  is  the  sum  of  squares  of  the  residuals  divided  by  the  degrees  of 
freedom.  The  degrees  of  freedom  value  is  [(NxM)  - (PxQ)].  Only  the 
model  variance  estimate  is  reproduced  here. 


Table  C2.  Terrain  Model  Error  Variances  (in  square  meters) 

Scene 


Model 

1 

2 

1 

1.03 

1.15 

2 

1.10 

1.24 

3 

0.93 

1.04 

4 

0.95 

1.15 

5 

0.00 

0.00 

3 

4 

5 

0.81 

0.84 

31.18 

0.90 

0.73 

49.36 

0.70 

0.82 

31.49 

0.62 

0.86 

15.92 

0.00 

0.00 

0.00 

The  zero  model  error  for  Model  5 is  because  there  were  as  many  unknown 
parameters  (256)  as  there  were  equations.  In  order  to  determine  how  well 
Model  5 actually  represents  the  data,  residuals  were  computed  for  the 
entire  set  of  2,116  points. 

Model  Error  Using  Entire  Data  Set.  The  objective  here  is  to 
see  if  Model  5 causes  large  excursions  of  elevation  estimates  at  points 
between  the  data  points  used  for  the  fit.  The  residuals  were  computed 
simply  by  evaluating  the  derived  function  at  the  2,116  points.  Note  that 
the  256  points  used  in  the  fit  will  necessarily  have  zero  residuals.  A 
summary  of  the  errors  is  given  in  table  C3. 
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Table  3C. 

Terrain 

Model  Five  Error  Summary 

(in  me 

ters) 

1 

Scene 

O 

3 

4 

5 

e 

0.0 

0.1 

0.1 

0.0 

-0.6 

°e 

0.88 

1.00 

1.04 

0.78 

5.74 

emax 

3.9 

4.3 

5.9 

3.3 

20.5 

emin 

-7.8 

-3.8 

-7.3 

■4.6 

-37.9 

Range 

11.7 

8.1 

13.2 

7.9 

58.4 

These  data  pertain  to  the  1,860  points  not  included  in  the  fit. 

Cumulative  probabilities  for  the  1,860  residuals  were  calculated  for 
this  example.  The  cumulative  tables  represent  the  distribution  of  errors 
over  the  range  of  errors.  The  error  range  in  this  example  was  quantized 
into  21  equal  intervals  over  the  range.  Note  that  the  range  of  errors 
differs  for  each  scene.  The  entries  in  table  C4  are  probability  (prob) 

(e  < O,  where  c is  an  error  and  £ is  the  right  side  of  the  £th  interval. 


Table  C4.  Terrain  Model  Five  Error  Probability  Function 


Scene 

L 

1 

2 

3 

4 

5 

l 

.0005 

.0022 

.0005 

.0005 

.0016 

2 

.0011 

.0054 

.0011 

.0011 

.0027 

3 

.0011 

.0102 

.0011 

.0011 

.0059 

4 

.0022 

.0156 

.0016 

.0011 

.0075 

5 

.0022 

.0312 

.0016 

.0016 

.0129 

6 

.0027 

.0516 

.0038 

.0038 

.0145 

7 

.0027 

.0919 

.0051 

.0102 

.0188 

8 

.0038 

. 1737 

.0145 

.0237 

.0280 

9 

.0078 

.3054 

.0333 

.0629 

.0414 

10 

.0129 

.4962 

.1016 

.1371 

.0548 

11 

.0301 

.6753 

.2780 

.2645 

.0699 

12 

.0720 

.8059 

.6317 

.4812 

. 1070 

13 

.2204 

.8941 

.8397 

.6914 

.2495 

14 

.4968 

.9371 

.9382 

.8468 

.6774 

15 

. 7935 

.9591 

.9683 

.9258 

.8914 

16 

.9328 

.9828 

.9866 

.9710 

.9511 

17 

.9785 

.9941 

.9898 

.9860 

.9763 
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Table  C4.  (Cont'd) 


18 

.9919 

.9962 

.9935 

.9919 

.9887 

19 

.9983 

.9973 

.9957 

.9962 

.9930 

20 

.9989 

.9984 

.9978 

.9989 

.9962 

21 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

The  cumulative  probabilities  and  the  interval  definitions  can  be  used  to 
compute  confidence  intervals  for  Model  5.  For  example,  99  percent 
confidence  intervals  for  the  five  scenes  are 


Scene 

1: 

Prob  [-2.8  <_  ej  <_  2.8] 

s 

.99 

Scene 

2: 

Prob  [-3.0  <_  C£  ± 3.1] 

- 

.99 

Scene 

3: 

Prob  [-2.9  £ £3  <_  4.6] 

- 

.99 

Scene 

4: 

Prob  [-2.7  C4  < 2.2] 

- 

.99 

Scene 

5: 

Prob  [-29.6  <_  C5  < 17. 

7] 

* . 

Consequently,  Model  5 represents  the  terrain  quite  well  for  the  first 
four  scenes  and  not  well  at  all  for  Scene  5. 

Discussion.  A portion  of  the  error  given  above  is  a compilation 
error,  that  is  errors  are  introduced  into  the  data  set  by  errors  in  the  I 

matching  process.  Scene  5 represents  a rugged  area;  wherein,  the  average 
correlation  was  low  compared  to  the  other  four  scenes.  Most  of  the  errors 
occurred  at  and  around  the  peak  in  the  upper  left  corner  of  the  scene,  an 
area  of  low  contrast.  It  is  probably  more  correct  to  say  that  the 
elevation  data  for  Scene  5 needs  to  be  refined  than  it  is  to  say  that  the 
model  is  deficient. 

Utilization  of  array  algebra  will  then  provide  the  analyst  with  - 
means  to  detect  rogue  measurements  as  well  as  a compaction  device.  For 

example,  if  Model  5 is  used,  then  a reduction  of  over  8 to  1 is  achieved  . 

with  compaction  errors  given  above.  The  bilinear  model  (not  necessarily  I 

the  ones  used  in  this  experiment)  can  be  used  in  compilation.  If  the 
elevation  data  is  represented  by  a bilinear  form,  then  its  coefficients 
can  be  developed  in  an  iterative  process;  wherein,  the  function  itself  is 
used  to  predict  matches  and  to  perform  complex  image  shaping.  The 
function  in  this  case  is  a terrain  model  and  contour  segments  can  be  devel- 
oped along  with  the  compilation  process  and  error  detection. 

Array  algebra  techniques  can  also  be  used  to  warp  one  image  to 
another.  If  the  dependent  pixel  data  is  represented  by  a bilinear  form, 
then  its  coefficients  can  be  developed  in  an  iterative  manner;  where  as 
before, the  function  is  used  for  predicting,  error  detecting,  and  shaping. 
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Image  Warp  Experiment . A simple  test  of  the  image  warp 
capability  of  array  algebra  was  performed  using  the  corresponding  imagery 
for  the  five  scenes  described  above.  Note  that  the  data  was  scanned  so 
that  the  epipolar  directions  were  nearly  parallel  to  the  pixel  Y-axis. 

For  this  reason,  the  following  model  was  fit  to  the  image  points 
corresponding  to  the  elevation  points  of  the  terrain  model  tests: 


V (X, Y)  = f (X)  A gT  (Y) 
where 


f (X)  = [1,  X,  X2,  X3,  SinFxX,  CosFxX] 


g (Y)  = [1,  Y,  Y2,  Y3,  SinFyY , Sin2F  Y,  Sin3F  Y , 

Sin4FyY,  CosFyY,  Cos2FyY,  Cos3FyY,  Cos4FyY] 


V (X,Y)  : Dependent  coordinate  in  epipolar  direction 

A:  (6  x 12)  array  of  unknown  coefficients 


A summary  of  the  errors  is  given  in  table  C5;  wherein,  the  units  are  pixel 
spacing.  The  statistical  results  pertain  to  all  2,116  points. 


Table  C5 . Warp  Model  Error  Summary 
Scene 


1 

2 

3 

4 

5 

e 

0.0 

0.0 

0.0 

0.0 

-0.2 

°e 

0.62 

0.67 

0.49 

0.46 

2.79 

emax 

2.6 

3.2 

2.5 

2.5 

12.2 

er.in 

-4.3 

-3.  3 

-3.3 

-2.3 

-18.7 

Range 

6.9 

6.5 

5.8 

4.8 

30.9 

umulative  probabilities  were 

computed 

as  before  and 

are  presented  in 

table  C6. 
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Table 

1 

C6.  Warp 
2 

Error  Probability 
Scene 

3 

Function 

4 

5 

-M. 

1 

.0009 

.0009 

.0009 

.0005 

.0005 

2 

.0009 

.0019 

.0009 

.0014 

.0024 

3 

.0014 

.0019 

.0009 

.0019 

.0024 

4 

.0019 

.0033 

.0028 

.0047 

.0033 

5 

.0028 

.0090 

.0038 

.0109 

.0071 

6 

.0052 

.0208 

.0071 

.0265 

.0132 

7 

.0080 

.0383 

.0113 

.0714 

.0213 

8 

.0132 

.0846 

.0184 

.1337 

.0312 

9 

.0250 

.1744 

.0401 

.2698 

.0430 

10 

.0525 

.3502 

.0912 

.4798 

.0666 

11 

.0997 

.5836 

.2264 

.6933 

.1224 

12 

.2164 

.7708 

.5208 

.8549 

.2533 

13 

.4556 

.8757 

.7717 

.9466 

.6243 

14 

.7108 

.9291 

.9041 

.9797 

.8549 

15 

.8842 

.9627 

.9542 

.9896 

.9386 

16 

.9527 

.9872 

.9853 

.9939 

.9778 

17 

.9783 

.9943 

.9934 

.9972 

.9915 

18 

.9929 

.9972 

.9948 

.9989 

.9962 

19 

.9972 

.9995 

.9953 

.9991 

.9981 

20 

.9990 

.9995 

.9991 

.9995 

.9991 

21 

1.0000 

1.0000 

1.0000  1 

.0000 

1.0000 

The  cumulative  probabilities  and  the  interval  definitions  can  be  used  to 
compute  confidence  intervals  for  the  model  above  when  used  as  a warp 
function.  For  example,  99  percent  confidence  intervals  for  the  five 
scenes  are 


Scene 

1: 

Prob 

1-2.0  1 ei  1 2.3]  = . 

99 

Scene 

2: 

Prob 

[-1. 1 £ Ej  < 2.6]  = . 

99 

Scene 

3: 

Prob 

[-1 . 9<C2<l-7}=* 

99 

Scene 

4 : 

Prob 

[-1.4  <_  e,  <_  1.4]  * . 

99 

Scene 

5: 

Prob 

[-11.3  < e5  < 9.3]  = 

.99 

units 

in  these 

intervals 

are  pixels.  Just  as 

in 

image 

warp  function  does 

not  fit  Scene  5 very 

we 
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